{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "735ae737-86be-4497-a8e9-38525e422380",
   "metadata": {},
   "source": [
    "# Semantic Search of one's own data with OpenAI Embedding Model and MongoDB Atlas Vector Search\n",
    "\n",
    "It is often a valuable exercise, when developing and documenting, to consider [User Stories](https://www.atlassian.com/agile/project-management/user-stories). We have a number of different personas interested in the ChatGPT Retrieval Plugin.\n",
    "\n",
    "1. The End User, who wishes to extract information from her organization's or personal data.\n",
    "2. The Data Scientist, who curates the data.\n",
    "3. The Application Engineer, who sets up and maintains the application.\n",
    "\n",
    "### Application Setup\n",
    "\n",
    "**The Application Engineer** has a number of tasks to complete in order to provide service to her two users.\n",
    "\n",
    "1. Set up the DataStore.\n",
    "\n",
    "\n",
    "    * Create a MongoDB Atlas cluster.\n",
    "    * Add a Vector Index Search to it.<br><br>\n",
    "\n",
    "    Begin by following the detailed steps in **[setup.md](https://github.com/caseyclements/chatgpt-retrieval-plugin/blob/mongodb/docs/providers/mongodb/setup.md)**.\n",
    "    Once completed, you will have a running Cluster, with a Database, a Collection, and a Vector Search Index attached to it.\n",
    "\n",
    "    You will also have a number of required environment variables. These need to be available to run this example.\n",
    "    We will check for them below, and suggest how to set them up with an `.env` file if that is your preference.\n",
    "\n",
    "  \n",
    "2. Create and Serve the ChatGPT Retrival Plugin.\n",
    "    * Provide an API for the Data Scientist to insert, update, and delete data.\n",
    "    * Provide an API for the End User to query the data using natural language.<br><br>\n",
    "    \n",
    "    Start the service in another terminal as described in the repo's **[QuickStart]( [here](https://github.com/openai/chatgpt-retrieval-plugin#quickstart)**. \n",
    "\n",
    "   **IMPORTANT** Make sure the environment variables are set in the terminal before `poetry run start`.\n",
    "\n",
    "### Application Usage\n",
    "\n",
    "This notebook tells a story of a **Data Scientist** and an **End User** as they interact with the service.\n",
    "\n",
    "\n",
    "We begin by collecting and fiiltering an example dataset, the Stanford Question Answering Dataset (SQuAD)[https://huggingface.co/datasets/squad].\n",
    "We upsert the data into a MongoDB Collection via the `query` endpoint of the Plugin API. \n",
    "Upon doing this, Atlas begins to automatically index the data in preparation for Semantic Search. \n",
    "\n",
    "We close by asking a question of the data, searching not for a particular text string, but using common language.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42f2c141-2643-4bff-b431-532916dfedf9",
   "metadata": {},
   "source": [
    "## 1) Application Engineering\n",
    "\n",
    "Of course, we cannot begin until we test that our environment is set up.\n",
    "\n",
    "### Check environment variables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02c045c9-39c8-47e4-a726-7b2a4c1cef21",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "143c3e90-cf24-45dc-af65-646bcf89b071",
   "metadata": {},
   "outputs": [],
   "source": [
    "!which python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8cfe7dc4-820c-4117-bdb7-debf3f5ec5ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "required_vars = {'BEARER_TOKEN', 'OPENAI_API_KEY', 'DATASTORE', 'EMBEDDING_DIMENSION', 'EMBEDDING_MODEL',\n",
    "                 'MONGODB_COLLECTION', 'MONGODB_DATABASE', 'MONGODB_INDEX', 'MONGODB_URI'}\n",
    "assert os.environ[\"DATASTORE\"] == 'mongodb'\n",
    "missing = required_vars - set(os.environ)\n",
    "if missing:\n",
    "    print(f\"It is strongly recommended to set these additional environment variables. {missing}=\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "88f07920-3616-4663-bb59-68179c97933e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you keep the environment variables in a .env file, like that .env.example, do this:\n",
    "if missing:\n",
    "    from dotenv import dotenv_values\n",
    "    from pathlib import Path\n",
    "    import os\n",
    "    config = dotenv_values(Path('../.env'))\n",
    "    os.environ.update(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0e152ba-4cb4-4703-ac30-035afbc84e67",
   "metadata": {},
   "source": [
    "### Check MongoDB Atlas Datastore connection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1aa90750-3f23-4671-8419-53e867649a6b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pinged your deployment. You successfully connected to MongoDB!\n"
     ]
    }
   ],
   "source": [
    "from pymongo import MongoClient\n",
    "client = MongoClient(os.environ[\"MONGODB_URI\"])\n",
    "# Send a ping to confirm a successful connection\n",
    "try:\n",
    "    client.admin.command('ping')\n",
    "    print(\"Pinged your deployment. You successfully connected to MongoDB!\")\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "52480c9d-fcf0-4cd2-8031-94999a0f87cc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Beyonce'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = client[os.environ[\"MONGODB_DATABASE\"]]\n",
    "clxn = db[os.environ[\"MONGODB_COLLECTION\"]]\n",
    "clxn.name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e334b80-babe-414d-adb1-6c4b8baff137",
   "metadata": {},
   "source": [
    "### Check OpenAI Connection\n",
    "\n",
    "These tests require the environment variables: `OPENAI_API_KEY, EMBEDDING_MODEL`\n",
    "\n",
    "We set the api_key, then query the API for its available models. We then loop over this list to find which can provide text embeddings, and their natural, full, default dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8435ce2e-ed38-48e8-a9eb-4595d9c8eee3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"model_dimensions={'text-embedding-3-small': 1536, 'text-embedding-ada-002': 1536, 'text-embedding-3-large': 3072}\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import openai\n",
    "openai.api_key = os.environ[\"OPENAI_API_KEY\"]\n",
    "models = openai.Model.list()\n",
    "model_names = [model[\"id\"] for model in models['data']]\n",
    "model_dimensions = {}\n",
    "for model_name in model_names:\n",
    "    try:\n",
    "        response = openai.Embedding.create(input=[\"Some input text\"], model=model_name)\n",
    "        model_dimensions[model_name] = len(response['data'][0]['embedding'])\n",
    "    except:\n",
    "        pass\n",
    "f\"{model_dimensions=}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcddf850-3a7c-4164-8862-88553d7b3970",
   "metadata": {},
   "source": [
    "## 2) Data Engineering\n",
    "\n",
    "### Prepare personal or organizational dataset\n",
    "\n",
    "The ChatGPT Retrieval Plug provides semantic search of your own data using OpenAI's Embedding Models and MongoDB's Vector Datastore and Semantic Search.\n",
    "\n",
    "In this example, we will use the **S**tanford **Qu**estion **A**nswering **D**ataset (SQuAD), which we download from Hugging Face Datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6833a874-e6f8-4f7b-9889-9b5184f458aa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "len(data)=19029\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>56be85543aeaaa14008c9063</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>When did Beyonce start becoming popular?</td>\n",
       "      <td>{'text': ['in the late 1990s'], 'answer_start'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>56be86cf3aeaaa14008c9076</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Following the disbandment of Destiny's Child i...</td>\n",
       "      <td>After her second solo album, what other entert...</td>\n",
       "      <td>{'text': ['acting'], 'answer_start': [207]}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>56be88473aeaaa14008c9080</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>A self-described \"modern-day feminist\", Beyonc...</td>\n",
       "      <td>In her music, what are some recurring elements...</td>\n",
       "      <td>{'text': ['love, relationships, and monogamy']...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>56be892d3aeaaa14008c908b</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles was born in Houston, T...</td>\n",
       "      <td>Beyonce's younger sibling also sang with her i...</td>\n",
       "      <td>{'text': ['Destiny's Child'], 'answer_start': ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>56be8a583aeaaa14008c9094</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé attended St. Mary's Elementary School ...</td>\n",
       "      <td>What town did Beyonce go to school in?</td>\n",
       "      <td>{'text': ['Fredericksburg'], 'answer_start': [...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          id    title  \\\n",
       "0   56be85543aeaaa14008c9063  Beyoncé   \n",
       "15  56be86cf3aeaaa14008c9076  Beyoncé   \n",
       "27  56be88473aeaaa14008c9080  Beyoncé   \n",
       "39  56be892d3aeaaa14008c908b  Beyoncé   \n",
       "52  56be8a583aeaaa14008c9094  Beyoncé   \n",
       "\n",
       "                                              context  \\\n",
       "0   Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "15  Following the disbandment of Destiny's Child i...   \n",
       "27  A self-described \"modern-day feminist\", Beyonc...   \n",
       "39  Beyoncé Giselle Knowles was born in Houston, T...   \n",
       "52  Beyoncé attended St. Mary's Elementary School ...   \n",
       "\n",
       "                                             question  \\\n",
       "0            When did Beyonce start becoming popular?   \n",
       "15  After her second solo album, what other entert...   \n",
       "27  In her music, what are some recurring elements...   \n",
       "39  Beyonce's younger sibling also sang with her i...   \n",
       "52             What town did Beyonce go to school in?   \n",
       "\n",
       "                                              answers  \n",
       "0   {'text': ['in the late 1990s'], 'answer_start'...  \n",
       "15        {'text': ['acting'], 'answer_start': [207]}  \n",
       "27  {'text': ['love, relationships, and monogamy']...  \n",
       "39  {'text': ['Destiny's Child'], 'answer_start': ...  \n",
       "52  {'text': ['Fredericksburg'], 'answer_start': [...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from datasets import load_dataset\n",
    "data = load_dataset(\"squad_v2\", split=\"train\")\n",
    "data = data.to_pandas().drop_duplicates(subset=[\"context\"])\n",
    "print(f'{len(data)=}')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c69543-51d7-49f9-b936-b7b99804818c",
   "metadata": {},
   "source": [
    "To speed up our example, let's focus specifically on questions about Beyoncé"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b444b0f8-a341-4f65-ab26-0793300d275f",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data.loc[data['title']=='Beyoncé']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2136e910-9944-422d-aaa5-50f1d3d7a5ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': '56be85543aeaaa14008c9063',\n",
       " 'text': 'Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ bee-YON-say) (born September 4, 1981) is an American singer, songwriter, record producer and actress. Born and raised in Houston, Texas, she performed in various singing and dancing competitions as a child, and rose to fame in the late 1990s as lead singer of R&B girl-group Destiny\\'s Child. Managed by her father, Mathew Knowles, the group became one of the world\\'s best-selling girl groups of all time. Their hiatus saw the release of Beyoncé\\'s debut album, Dangerously in Love (2003), which established her as a solo artist worldwide, earned five Grammy Awards and featured the Billboard Hot 100 number-one singles \"Crazy in Love\" and \"Baby Boy\".',\n",
       " 'metadata': {'title': 'Beyoncé'}}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents = [\n",
    "    {\n",
    "        'id': r['id'],\n",
    "        'text': r['context'],\n",
    "        'metadata': {\n",
    "            'title': r['title']\n",
    "        }\n",
    "    } for r in data.to_dict(orient='records')\n",
    "]\n",
    "documents[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98388d9a-bb33-4eea-8d43-e890517f829a",
   "metadata": {},
   "source": [
    "## Upsert and Index data via the Plugin API\n",
    "\n",
    "Posting an `upsert` request to the ChatGPT Retrieval Plugin API performs two tasks on the backend. First, it inserts into (or updates) your data in the MONGODB_COLLECTION in the MongoDB Cluster that you setup. Second, Atlas asynchronously begins populating a Vector Search Index on the embedding key. \n",
    "\n",
    "If you have already created the Collection and a Vector Search Index through the Atlas UI while Setting up MongoDB Atlas Cluster in [setup.md](https://github.com/caseyclements/chatgpt-retrieval-plugin/blob/main/docs/providers/mongodb/setup.md), then indexing will begin immediately.\n",
    "\n",
    "If you haven't set up the Atlas Vector Search yet, no problem. `upsert` will insert the data. To start indexing, simply go back to the Atlas UI and add a Search Index. This will trigger indexing. Once complete, we can begin semantic queries!\n",
    "\n",
    "\n",
    "The front end of the Plugin is a FastAPI web server. It's API provides simple `http` requests.'We will need to provide authorization in the form of the BEARER_TOKEN we set earlier. We do this below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8e2891fb-4485-4f97-b73f-e2f4186238bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_url = 'http://0.0.0.0:8000'\n",
    "headers = {\"Authorization\": f\"Bearer {os.environ['BEARER_TOKEN']}\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f5cc603-88f1-402b-a0a7-52cabb9f7d9d",
   "metadata": {},
   "source": [
    "Although our sample data is not large, and the service and datastore are reponsive, we follow best-practice and execute bulk upserts in batches with retries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1c0e2eb7-58b5-4f36-af2d-d29aee18e08d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb28c6a61b464858806fc8f3404e7a19",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(i,i_end) =(0, 66)\n"
     ]
    }
   ],
   "source": [
    "from tqdm.auto import tqdm\n",
    "import requests\n",
    "from requests.adapters import HTTPAdapter, Retry\n",
    "\n",
    "# Setup request parameters to batch requests and retry on 5xx errors\n",
    "batch_size = 100\n",
    "retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])\n",
    "session = requests.Session()\n",
    "session.mount('http://', HTTPAdapter(max_retries=retries))\n",
    "n_docs = len(documents)\n",
    "for i in tqdm(range(0, n_docs, batch_size)):\n",
    "    i_end = min(n_docs, i+batch_size)\n",
    "    print(f'{(i,i_end) =}') \n",
    "    # make post request that allows up to 5 retries\n",
    "    res = session.post(\n",
    "        f\"{endpoint_url}/upsert\",\n",
    "        headers=headers,\n",
    "        json={\"documents\": documents[i:i_end]}\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6f193886-f14e-4d97-9fc8-6c269b67602f",
   "metadata": {},
   "outputs": [],
   "source": [
    "if res.status_code != 200:\n",
    "    res.text, res.reason"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "938f4f7c-5a67-4f54-9596-cf27a79e05e8",
   "metadata": {},
   "source": [
    "# 3) Answering Questions\n",
    "\n",
    "Now would be a good time to go back to the Atlas UI, navigate to your collection's search index. Once all our SQuAD records have been successfully indexed, we can proceed with the querying phase. By passing one or more queries to the /query endpoint, we can easily conduct a query on the datastore. For this task, we can utilize a few questions from SQuAD2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1f0a9973-1326-4d16-9c47-c5245675ea44",
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_results(results):\n",
    "    for query_result in results.json()['results']:\n",
    "        query = query_result['query']\n",
    "        answers = []\n",
    "        scores = []\n",
    "        for result in query_result['results']:\n",
    "            answers.append(result['text'])\n",
    "            scores.append(round(result['score'], 2))\n",
    "        print(\"-\"*70+\"\\n\"+query+\"\\n\\n\"+\"\\n\".join([f\"{s}: {a}\" for a, s in zip(answers, scores)])+\"\\n\"+\"-\"*70+\"\\n\\n\")    \n",
    "\n",
    "def ask(question: str):\n",
    "    res = requests.post(\n",
    "        f\"{endpoint_url}/query\",\n",
    "        headers=headers,\n",
    "        json={'queries': [{\"query\": question}]}\n",
    "    )\n",
    "    format_results(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b257e50b-f888-49c8-9e12-c98b8726aaab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------\n",
      "Who is Beyonce?\n",
      "\n",
      "0.83: Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ bee-YON-say) (born September 4, 1981) is an American singer, songwriter, record producer and actress. Born and raised in Houston, Texas, she performed in various singing and dancing competitions as a child, and rose to fame in the late 1990s as lead singer of R&B girl-group Destiny's Child. Managed by her father, Mathew Knowles, the group became one of the world's best-selling girl groups of all time. Their hiatus saw the release of Beyoncé's debut album, Dangerously in Love (2003), which established her as a solo artist worldwide, earned five Grammy Awards and featured the Billboard Hot 100 number-one singles \"Crazy in Love\" and \"Baby Boy\".\n",
      "0.82: A self-described \"modern-day feminist\", Beyoncé creates songs that are often characterized by themes of love, relationships, and monogamy, as well as female sexuality and empowerment. On stage, her dynamic, highly choreographed performances have led to critics hailing her as one of the best entertainers in contemporary popular music. Throughout a career spanning 19 years, she has sold over 118 million records as a solo artist, and a further 60 million with Destiny's Child, making her one of the best-selling music artists of all time. She has won 20 Grammy Awards and is the most nominated woman in the award's history. The Recording Industry Association of America recognized her as the Top Certified Artist in America during the 2000s decade. In 2009, Billboard named her the Top Radio Songs Artist of the Decade, the Top Female Artist of the 2000s and their Artist of the Millennium in 2011.\n",
      "0.79: When Beyoncé does an album, when Beyoncé sings a song, when Beyoncé does anything, it's an event, and it's broadly influential. Right now, she is the heir-apparent diva of the USA — the reigning national voice.\" In 2014, Beyoncé was listed again on the Time 100 and also featured on the cover of the issue.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "ask(\"Who is Beyonce?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c36d54ae-5bac-4ceb-b98d-6067867f954c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------\n",
      "Who is Beyonce married to?\n",
      "\n",
      "0.78: On April 4, 2008, Beyoncé married Jay Z. She publicly revealed their marriage in a video montage at the listening party for her third studio album, I Am... Sasha Fierce, in Manhattan's Sony Club on October 22, 2008. I Am... Sasha Fierce was released on November 18, 2008 in the United States. The album formally introduces Beyoncé's alter ego Sasha Fierce, conceived during the making of her 2003 single \"Crazy in Love\", selling 482,000 copies in its first week, debuting atop the Billboard 200, and giving Beyoncé her third consecutive number-one album in the US. The album featured the number-one song \"Single Ladies (Put a Ring on It)\" and the top-five songs \"If I Were a Boy\" and \"Halo\".\n",
      "0.77: Beyoncé is believed to have first started a relationship with Jay Z after a collaboration on \"'03 Bonnie & Clyde\", which appeared on his seventh album The Blueprint 2: The Gift & The Curse (2002). Beyoncé appeared as Jay Z's girlfriend in the music video for the song, which would further fuel speculation of their relationship. On April 4, 2008, Beyoncé and Jay Z were married without publicity. As of April 2014, the couple have sold a combined 300 million records together. The couple are known for their private relationship, although they have appeared to become more relaxed in recent years. Beyoncé suffered a miscarriage in 2010 or 2011, describing it as \"the saddest thing\" she had ever endured. She returned to the studio and wrote music in order to cope with the loss.\n",
      "0.76: In the same year, Beyoncé and Jay Z placed at number one on the \"World's Highest-Paid Celebrity Couples\", for collectively earning $78 million. The couple made it into the previous year's Guinness World Records as the \"highest-earning power couple\" for collectively earning $122 million in 2009. For the years 2009 to 2011, Beyoncé earned an average of $70 million per year, and earned $40 million in 2012. In 2013, Beyoncé's endorsements of Pepsi and H&M made her and Jay Z the world's first billion dollar couple in the music industry. That year, Beyoncé was published as the fourth most-powerful celebrity in the Forbes rankings. MTV estimated that by the end of 2014, Beyoncé would become the highest-paid black musician in history; she succeeded to do so in April 2014.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "ask(\"Who is Beyonce married to?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4231d114-6341-433d-9b82-a1527d64775c",
   "metadata": {},
   "source": [
    "## 4) Clean up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9d12f4d6-9205-4c2e-bf17-f84956065672",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'success': True}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.delete(\n",
    "    f\"{endpoint_url}/delete\",\n",
    "    headers=headers,\n",
    "    json={\"delete_all\":True}\n",
    ")\n",
    "\n",
    "response.json()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
